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1. Partial Least Squares for policy evaluation 


Structural Equation Modeling (SEM), especially Partial Least Squares - Path Modeling 
(PLS-PM) has become a mainstream method in many fields of research. Indeed, PLS-PM has 
been used in the social and behavioral sciences, rooted in psychometrics and in the literature on 
causal modeling. In the last few years it has been increasingly disseminated in a variety of dis- 
ciplines, and, in particular, has been extensively used in the business and management sciences. 
Within these research projects, PLS-PM has been applied successfully in studies concerning the 
measurement of intangibles like customer and employee perceptions (e.g. satisfaction, motiva- 
tion and loyalty). These kinds of model are becoming crucial to managers in order to improve 
their decision making processes and increase their organization’s profitability. In every time 
and place the decision making process has always been complex. Generally, it applies evalu- 
ation principles and methods to examine the content, implementation or impact of a policy or 
a decision. In the last few years, researchers have been promoting statistical methods such as 
SEM and PLS-PM for the evaluation of policies, especially in the context of decision making. 
In the literature, empirical approaches which have applied PLS-PM to decision making have 
been identified through a systematic literature search. To better understand and characterize 
this trend, a bibliometric study of international papers on this subject has been developed in 
order to describe the use of SEM and PLS-PM approaches in policy evaluation during the last 
twenty years. 


2. Study Methodology 


A bibliometric analysis has been used to analyse the trends in the field of SEM in the con- 
text of decision making. Bibliometric analysis is a quantitative approach for the analysis of 
academic literature using bibliographies to provide the description, evaluation and monitoring 
of the published research (Garfield et al., 1964); (White and McCain, 1989). The methodolog- 
ical aim is to analyse publications, citations and sources of information (Rodriguez-Soler et 
al., 2020). Bibliographic data are processed through a workflow: study design, data collection, 
data analysis, data visualization and interpretation. The analysis has been performed using the 
Bibliometrix R-Tool (Aria and Cuccurullo, 2017), a recent R-package which facilitates a more 
complete bibliometric analysis employing specific tools for both bibliometric and scientomet- 
ric quantitative research. The Bibliometrix R-package (http://www. bibliometrix.org) provides a 
set of tools for quantitative research in bibliometrics and scientometrics, supporting scholars in 
three key phases of analysis: 1) data importing and conversion to the R format; 2) bibliometric 
analysis of a publication dataset; and 3) building matrices for co-citation, coupling, collabo- 
ration and co-word analysis. The R program and the bibliometrix codes have been used to 
produce a descriptive bibliometric analysis and to construct the matrices. In addition, “bib- 
lioshiny” (Aria and Cuccurullo, 2017), a Bibliometrix web-interface, has been used to build a 
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conceptual map and network for co-citation. Matrices are the input data for the performance of 
network analysis, multiple correspondence analysis and certain data reduction techniques (Aria 
and Cuccurullo, 2017). 


3. Data Collection 


With the aim of understanding how the research on SEM and PLS-PM issues has evolved, 
the data were retrieved from two main databases commonly used by researchers: Scopus and 
Web of Science (WoS). Scopus and WoS are the world’s most trusted independent global ci- 
tation database. They are recognised as covering a broad range of relevant journals and peer- 
reviewed articles of high quality (Skute et al., 2019). These databases have already been used in 
bibliometric analysis in different disciplines, sometimes individually, in the case of WoS (Diem 
and Wolter, 2013); (Falagas et al. , 2006) and Scopus (Maharana, 2013); (Morandi et al., 2015), 
and sometimes in combination (Rodriguez-Soler et al., 2020). 

We extracted articles published between 2000 and 2020 (incl.) which contained the topic 
“decision making” with the following keywords in the title or abstract: “PLS-PM”; “PLS Path 
modeling”; “PLS-Path modeling”; “SEM-PLS” (“decision making” AND “PLS-PM” OR “PLS 
Path modeling” OR “PLS-Path modeling” OR “SEM-PLS”). The data were downloaded on 
December 5, 2020. Only articles, reviews, proceedings papers and book chapters were included, 
with document types such as editorials, notes and corrections were excluded from the study. By 
merging the Scopus and WoS databases, 93 duplicate documents were removed. This process 
resulted in a final sample of 451 articles, which constitute the core material of this study, relating 
to 1,308 authors and 323 sources. 


4. Analysis and Discussion 


In the analysis of the data, a descriptive analysis was initially performed. Next, bibliometric 
techniques were developed using conceptual, intellectual or social networks. 
Figure 1 shows the growth of publications from 2000 to 2020. 
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Figure 1: Growth trajectory of the literature relating to the use of PLS-PM in decision making, 
2008 - 2020 


As can we see, the first studies dealing with issues related to decision making were carried 
out in 2008. In the first years of the analysis (2008 - 2012) the number of publications is very 
low, emphasizing the fact that the topic was probably not very well developed and addressed 
by researchers. In 2019 we see a peak in the number of publications relating to the PLS-PM 


50 


approach as a Statistical methods in the context of decision making. 

Concerning the sources, the distribution of the articles does not present any significant con- 
centration. The journals which included the most frequently quoted articles, containing “deci- 
sion making” and “PLS-PM” as keywords, are presented in Figure 2, with the largest number 
of articles, namely 17, published in the Sustainability journal, followed by the International 
journal of environmental research and public health, a journal which deals with issues related 
to environmental health sciences and public health, measurement and monitoring models. 
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Figure 2: The most relevant sources 


As regards provenance, the research activity of countries in terms of their publication output 
on this theme was examined. Figure 3 shows the top 20 most productive countries in terms of 
publication output and scientific collaborations during the period 2008-2020. In particular, 
the left-hand side of the Figure 3 shows the number of articles produced by the authors of 
different countries and the rate of cooperation of each country’s authors with those of other 
countries, while the right-hand side of the Figure 3 shows the cooperations and networking 
among researchers working on and studying this subject in different countries. 


Figure 3: Country production and country collaboration networks 


The authors who have distinguished themselves in terms of the number of publications re- 
lated to this topic come mainly from China, Malaysia, the USA and Italy. The authors from 
China and Malaysia have produced the same numbers of articles (35) (not shown here), but the 
rate of Chinese authorship with other countries is about 46% while the rate of MCP (Multiple 
Country Pubblications) of Malaysia is 17%. This demonstates that Chinese authors collaborate 
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extensively with authors from other countries. Italian authors ranked third with 30 papers, and 
the authorship rate for contributing articles to other authors from other countries is 27.6%. As 
we can see in the ranking, Italy is the European country that has most significantly increased 
its publication output in relation to policy evaluation in recent years, indicating that Italian re- 
searchers have been promoting statistical methods such as SEM and PLS-PM for the evaluation 
of policies. 

The networking analysis emphasizes the strong collaboration from the Chinese researchers 
with those from countries such as the USA and Australia (the strength of the collaboration is 
indicated by the thickness of the links), while European researchers prefer to collaborate with 
each other. In particular, there is a strong collaboration between France, Spain and Italy. The 
size of the name of the country is related to the number of works published on the analyzed 
topic, while the different colors of the countries and of the links represent the clusters that have 
been formed, as determined by the program algorithm. 

Figure 4 highlights some of the most frequently used topics in studies associated with “deci- 
sion making” and “PLS-PM” during this period. As can be observed from Figure 4, topics relat- 
ing to evaluation start to appear in 2018 (“life”, “prevention”, “trasportation”). The frequency 
increases with the passing of the years. The words most commonly used by researchers who 


have applied PLS-PM in their studies during the last two years have been “students”, “educa- 


tion”, “university”, “perceptions”, “job”, “learning”, “growth” and “country”, topics associated 
with policy evaluation and decision making issues. 


Trend Topics 


log frequency) 


Figure 4: Trend of the topics over the time period 


The final figure, Figure 5, shows the keywords considered as themes, classified by different 
levels of density and centrality in the network of scientific keywords. In the strategic diagram 
presented in Figure 5, the vertical axis measures the density, namely the strength of the internal 
links within a cluster represented by a theme, and the horizontal axis the centrality, namely 
the strength of the internal links within a cluster represented by a theme, and the horizontal 
axis the centrality, namely the strength of the links between the theme and other themes in the 
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map (Pourkhani et al., 2019). A thematic map is a very intuitive plot, enabling an analysis of 
themes according to the quadrant in which they are placed, namely: (1) the upper-right quadrant: 
motor-themes; (2) the lower-right quadrant: basic themes (3) the lower-left quadrant: emerging 
or disappearing themes; (4) the upper-left quadrant: very specialized/niche themes (Cataldo et 
al., 2019). 
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Figure 5: Thematic map 


Author’s keywords linked to “satisfaction” appear as a motor theme, emphasizing how in the 
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last few years researchers have focused their attention on this theme; words like “trust”, “ser- 
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vice quality”, “customer loyalty”, “relationship” and “perceived value” appear in this cluster. 
Themes with a higher centrality include “pls-sem”, “pls-pm’”, topics that appear ubiquitously in 
different scientific works and can be considered a common synthesis of the content expressed 
in the literature. The topic “Malaysia” appears also in this quadrant, while “China”, “engage- 
ment”, “finantial performance” and “risk perception” are other author keywords presented in 
this cluster, highlighting the predominance of Malaysian and Chinese pubblications in relation 


to the evaluation theme. Keywords such as “consumer behaviour’, “decision-making”, “‘con- 
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sumer”, “crowfunding”, despite having a low centrality, have a higher frequency, showing that 
these themes are considered very specialized topics in these scientific works. 


5. Conclusion 


The decision making process has always been complex. In the last few years, researchers 
have been promoting the use of statistical methods such as SEM and PLS-PM for the evaluation 
of policies, especially in the context of decision making. To better understand and characterize 
the trend of the scientific pubblication relating to this theme, a bibliometric study of interna- 
tional papers on this subject has been developed highlighting the use of SEM and PLS-PM 
approaches in policy evaluation during the last twenty years. The data were retrieved from two 
main databases commonly used by researchers, Scopus and Web of Science, and the analysis 
of 451 articles was performed using bibliometrix R-Tool (Aria and Cuccurullo, 2017). The re- 
sults suggest that the interest in research on this topic has increased in recent years, particularly 
between 2015 and 2019, indicating that this issue has become a significant topic of attention 
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among researchers in this period. Globally, China is ranked first in terms of production, while 
in Europe Italian researchers are the most prominent in the promotion of statistical methods 
such as SEM and PLS-PM for policy evaluation, also collaborating with scholars in Spain and 
France. The words most frequently used in the last two years by researchers who deal with 
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PLS-PM in their studies have been “students”, “education”, “university”, “perceptions”, “job”, 
“learning”, “growth” and “country”, topics associated with policy evaluation and decision mak- 
ing issues. This study has analysed scientific pubblications on databases being constantly up- 
dated. Therefore, a bibliometric analysis regarding an emergent theme may, in a few years, 
be subject to substantial variations. Furthemore, the present study has analysed a particular 
theme using two different databases. Despite them being two of the most influential databases, 
the global perspective could be improved with the inclusion of other databases. However, the 
results obtained from this analysis may assist researchers in investigation this theme and in fo- 
cusing on developing the PLS-PM approach for policy evaluation and decision making in many 
fields of research. 
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